Many of us have seen films containing remarkably realistic dinosaurs, aliens, animated toys and other fanciful creatures. Such animations are made possible by computer graphics. Using such techniques, a computer graphics artist can specify how each object should look and how it should change in appearance over time, and a computer then models the objects and displays them on a display such as your television or a computer screen. The computer takes care of performing the many tasks required to make sure that each part of the displayed image is colored and shaped just right based on the position and orientation of each object in a scene, the direction in which light seems to strike each object, the surface texture of each object, and other factors.
Because computer graphics generation is complex, computer-generated three-dimensional graphics just a few years ago were mostly limited to expensive specialized flight simulators, high-end graphics workstations and supercomputers. The public saw some of the images generated by these computer systems in movies and expensive television advertisements, but most of us couldn't actually interact with the computers doing the graphics generation. All this has changed with the availability of relatively inexpensive 3D graphics platforms such as, for example, the Nintendo 64® and various 3D graphics cards now available for personal computers. It is now possible to interact with exciting 3D animations and simulations on relatively inexpensive computer graphics systems in your home or office.
A problem graphics system designers confronted in the past is how to avoid bad visual effects associated with aliasing in a displayed image. Most modern computer graphics display devices create images by displaying an array of colored dots called pixels. Home color television sets and computer monitors work this way. When displaying graphics images on this kind of pixelated display, a staircasing effect can result due to the inherent characteristics of the graphics system and the display. Because the displayed digital image is made up of an array or grid of tiny pixels, edges of objects in the image may look jagged or stepped. For example, a smooth edge may appear as a stepped or jagged line due to the pixel grid. People refer to this stepped or jagged edge effect as “the jaggies” or “staircasing”—but its technical name is “aliasing”.
Aliasing is an inherent feature of a sampling based system. An unpleasant image can result when jaggies exist along edges and intersections of rendered primitives. Moreover, other visually disturbing side-effects of aliasing such as texture “swimming” or “flickering” can result throughout the entire rendered scene. These annoying side-effects are most often noticeable during animation.
Much work has been done in the past to solve the aliasing problem. More expensive graphics systems use “anti-aliasing” techniques that reduce or eliminate the visual effects of aliasing. One common anti-aliasing technique is based on a super-sampling/postfiltering approach. Using this approach, the graphics system develops a sampled image that has more samples (sub-pixels) than the display device is capable of displaying. The graphics system filters the higher-resolution sampled image and resamples the image at the resolution of the display device. In simple terms, the graphics system intentionally coarsens the resolution of the sampled image before displaying it on the display device. In one example, the graphics system might generate a certain number of sub-pixels for a pixel to be displayed on the screen, and blend the sub-pixels together to create the corresponding screen pixel.
Such anti-aliasing techniques improve the appearance of the image by reducing jaggies. The blurring or blending of pixels smoothes out edges—even though the image is still made up of discrete pixels—because it provides a more gradual change in the pixel color pattern. As a result, the eye of the viewer perceives the edge as being much smoother and more accurate as compared to an aliased edge. It is not exactly intuitive that blurring could make the edge appear to be more accurate and realistic, but this is exactly how commonly used anti-aliasing techniques work.
Unfortunately, however, the super-sampling anti-aliasing approach described above requires a substantial amount of memory and other resources. For example, storing n sub-pixels for each screen pixel requires a memory that is n times the size of what would otherwise be required. In addition, generating n sub-pixels for each screen pixel requires the graphics pipeline to do a lot of extra work. Also, the blending operation can be very burdensome and can require additional circuitry or other processing resources. Consequently, such super-sampling approaches to anti-aliasing have typically been found in the past in expensive graphics systems such as high end workstations but have been too “expensive” (in terms of required processing and memory resources) for use in low cost systems such as video game platforms.
Another anti-aliasing technique that has been used in the past involved the use of coverage values to reduce computational complexity and memory requirements. Such technique is described in U.S. Pat. No. 5,742,277. This technique provides a method of anti-aliasing a silhouette edge by retrieving a color value for a silhouette edge pixel which falls on the silhouette edge from a frame buffer, the retrieved color value representing a color of one or more foreground polygons which fall within the silhouette edge pixel. The technique estimates a background color of the silhouette edge pixel based on colors of neighboring pixels that are proximate to the silhouette edge pixel. This estimated background color represents a color of a portion of the silhouette edge pixel which is not occupied by the one or more foreground polygons. An output color of the silhouette edge pixel is determined by interpolating between the retrieved color and the estimated background color. While this anti-aliasing technique can reduce jaggies in the rendered scene, the required estimation step has distinct disadvantages in terms of accuracy.
Another antialiasing approach is disclosed in U.S. Pat. Nos. 6,072,500 & 5,684,939. In this prior approach, a method for generating antialiased display data comprises storing a pixel memory that indicates a current state of a pixel that comprises a plurality of supersamples, wherein said pixel memory comprises a region mask having a plurality of fields, each field being associated with a unique one of said supersamples; receiving a pixel packet, wherein said pixel packet indicates polygon coverage within said pixel, and a first color value; storing a second color value in an image memory, wherein said second color value is a function of said first color value; determining a new pixel state based on said current pixel state and said pixel packet; updating said pixel memory based on said new pixel state, wherein if said new pixel state is a state in which the color value of each supersample is either said second color value or a third color value, each of the fields associated with a supersample having said second color value stores an identifier that identifies said image memory; and generating antialiased display data based on said pixel memory. One drawback with this technique is that it requires region masks to be stored in pixel memory—with a corresponding increase in the size and cost of the pixel memory.
In summary, although various full-scene anti-aliasing (FSAA) techniques have been developed to mitigate the aliasing problem with varying degrees of success, some of the more effective approaches (for example, those involving conventional super-sampling and per-pixel object-precision area sampling), are often too computationally intensive and expensive to implement within a low cost graphics system such as a home video game platform. Other techniques developed for lower cost systems have been partially effective, but suffer from accuracy problems. Therefore, while significant work has been done in the past, further improvements in anti-aliasing are desirable.
The present invention solves this problem by providing improved techniques and arrangements for anti-aliasing in a graphics system.
In accordance with one aspect of our invention, we have developed particular techniques for anti-aliasing using an embedded frame buffer. For example, we have discovered particularly advantageous ways to perform anti-aliasing on the fly during a “copy out” process wherein an image data representation is being transferred from an embedded frame buffer to another destination. Such techniques provide a highly efficient and cost-effective antialiasing approach that can be practically implemented in a low cost system.
We have also discovered ways to achieve higher anti-aliasing quality using a smaller number of multisamples than were formerly required. Typical existing multisample methods use “n” samples and a 1×1 box filter for reconstruction. We have discovered that by using a combination of particular sample patterns (i.e., multisample spatial distribution) and particular filter configurations that “share” some multisamples among several pixels, we can achieve better antialiasing than an “n” sample pattern and a reconstruction/antialiasing filter that extends across only single pixel area.
For example, using three multisamples per pixel and a 1×2 reconstruction filter (i.e., a vertical filter that extends into one-half of the neighboring pixel areas immediately above and below the current pixel), and by using a specific sample pattern, we are able to achieve the equivalent of 6-sample antialiasing on vertical edges. Similarly, using a 1.33×2 reconstruction filter and a different sampling pattern, we achieve the equivalent of 6-sample antialiasing on vertical edges and 4-sample antialiasing on horizontal edges. On a more intuitive level, we are intentionally varying (jittering) the sample pattern between pixels so as to achieve better antialiasing at the expense of noise along the edges; and then increasing the extent of the reconstruction filter to greater than 1×1 to reduce or eliminate the additional noise while sharing some multisamples between pixels for antialiasing purposes—thus achieving the effect of more multisamples than we are actually storing on a per-pixel basis in the frame buffer. This dramatic increase in antialiasing quality without requiring a corresponding increase in the number of multisamples stored in the frame buffer has particular advantages for low-cost graphics systems such as home video game platforms and personal computer graphics cards.
Thus, in accordance with one aspect of the invention, a graphics system including graphics circuits coupled to an embedded frame buffer renders a multisampled data representation of an image and stores the rendered multisampled data representation in the embedded frame buffer. We then resample said embedded frame buffer contents to provide an anti-aliased image. We can perform such antialiasing filtering on the image in the process of transferring the image from the embedded frame buffer to another location.
In accordance with yet another aspect of the invention, an anti-aliasing method implemented within a graphics system of the type that generates an image comprising plural pixels involves generating a multisampled data representation of an image having plural samples associated with each of the plural pixels. We resample the multisampled data representation to create an antialiased image for display. The resampling includes blending at least one of the plural samples into plural image pixels (i.e., sharing some of the multisamples between plural reconstructed screen pixels).
In accordance with yet another aspect provided by the invention, an anti-aliasing method comprises providing plural supersamples within each pixel of a pixel array. We vary the spatial distribution of the supersamples within neighboring pixels of the pixel array, and apply, to the array, an anti-aliasing filter having a pixel aperture including supersamples of at least two neighboring pixels.
In accordance with a more detailed aspect provided by the invention, an anti-aliasing method comprises:                defining, within an embedded frame buffer, plural (e.g., three) super-sampled locations within each pixel of a pixel array, each said super-sampled location having a corresponding color value; and        applying a vertical color data blending filter that blends a set of the pixel super-sampled color values during an operation that copies the embedded frame buffer out to an external destination.By way of further non-limiting example, the following are some of the additional features provided by aspects of the invention:        coverage masking of programmable super-sample locations efficiently generate a super-sampled image;        a one-dimensional (e.g., vertical) filter applied during a copy-out operation from an embedded frame buffer to an external frame buffer can be used to blend the super-sampled image;        super-samples from neighboring pixels can be included in the anti-aliased blend; and        programmable locations and filtering weight(s) of the supersamples in the blend.        
Another aspect of the invention provides, in a graphics system, a pixel data processing arrangement for providing full-scene anti-aliasing and/or de-flickering interlaced displays, comprising:                a frame buffer containing super-sampled pixel data for a plurality of pixels;        a plurality of scan-line buffers connected to receive super-sampled pixel color data from the frame buffer; and        a multi-tap selectable-weight blending filter coupled to the scan-line buffers, the blending filter characterized by a vertically-arranged multiple-pixel filter support region wherein one or more color data samples from a plurality of vertically disposed pixels are blended to form a pixel color.        
A further example anti-aliasing arrangement provided in accordance with an aspect of the invention includes:                at least one storage location that defines plural super-sample locations within at least one pixel of a pixel array, each super-sample location having a corresponding color value;        a coverage mask that specifies, for each of the plural super-sample locations within the at least one pixel, whether the plural super-sample locations are covered by rendered primitive fragments; and        a one-dimensional color data blending filter that blends a resulting set of super-sample color values based on a programmable weighting function.        
In one particular, non-limiting arrangement, the storage location may define three super-sample locations within the pixel. The filter may blend super-sample color values corresponding to the pixel with super-sample color values corresponding to at least one further pixel neighboring the pixel. The filter may blend super-sample color values corresponding to three vertically aligned pixels to produce a screen pixel output.
A further particular anti-aliasing technique provided in accordance with an aspect of the invention operates by:                defining three sample locations for obtaining super-sampled color data associated with a pixel for each of a plurality of neighboring pixels;        using a coverage mask to enable/disable samples corresponding to such locations, the coverage mask being based at least in part on corresponding portions of each pixel that are occupied by rendered primitive fragments; and        blending resulting color data obtained from the locations to provide a pixel final color value.        
A further aspect of the invention provides, for a pixel quad having first, second, third and fourth pixels and a quad center, a method of defining an optimal set of three super-sampling locations for anti-aliasing comprising:                defining a first set of super-sample locations for a first pixel in the pixel quad at the following coordinates (range 1–12) relative to the quad center: (12,11) (4,7) (8,3);        defining a second set of super-sample locations for a second pixel in the pixel quad at the following coordinates (range 1–12) relative to the quad center: (3,11) (11,7) (7,3);        defining a third set of super-sample locations for a third pixel in the pixel quad at the following coordinates (range 1–12) relative to the quad center: (2,2) (10,6) (6,10); and        defining a fourth set of super-sample locations for a fourth pixel in the pixel quad at the following coordinates (range 1–12) relative to the quad center: (9,2) (1,6) (5,6).        
In still more detail, a preferred embodiment of the present invention provides efficient full-scene anti-aliasing by, inter alia, implementing a programmable-location super-sampling arrangement and using a selectable-weight vertical-pixel support area blending filter. For a 2×2 pixel group (quad), the locations of three samples within each super-sampled pixel are individually selectable. Preferably, a twelve-bit multi-sample coverage mask is used to determine which of twelve samples within a pixel quad are enabled based on the portions of each pixel occupied by a primitive fragment and any pre-computed z-buffering. Each super-sampled pixel is filtered during a copy-out operation from a local memory to an external frame buffer using a pixel blending filter arrangement that combines seven samples from three vertically arranged pixels. Three samples are taken from the current pixel, two samples are taken from a pixel immediately above the current pixel and two samples are taken from a pixel immediately below the current pixel. A weighted average is then computed based on the enabled samples to determine the final color for the pixel. The weight coefficients used in the blending filter are also individually programmable. De-flickering of thin one-pixel tall horizontal lines for interlaced video displays can be accomplished by using the pixel blending filter to blend color samples from pixels in alternate scan lines.